MEMORANDUM February 10, 2016 


TO: Board Members 


FROM: Terry B. Grier, Ed.D. 
Superintendent of Schools 


SUBJECT: Head Start Program Longitudinal Effects Study: Third Grade Follow-Up, 
2014-2015 Third Grade STAAR Reading and Mathematics Performance 


CONTACT: Carla Stevens, (713) 556-6700 


The purpose of this report was to evaluate the third grade academic performance of a cohort of 
students who attended one of the four Head Start programs in 2010-2011. The 2014-2015 
third grade STAAR (regular English version) reading and mathematics tests were used as the 
outcome measures to assess the lasting impact of Head Start programs on students’ academic 
performance. 


Key findings include: 

e Economically-disadvantaged students who attended Head Start programs obtained higher 
mean scale scores, and had higher percentages of students who met the 2015 STAAR 
Level Il: Satisfactory (Phase-In 1) standard on both third grade reading and mathematics 
tests than their peers in the district. 

e Comparisons by four Head agencies showed that Neighborhood Centers, Inc. (NCI) had 
the highest percentage of students who met the 2015 STAAR Level II: Satisfactory 
(Phase-In 1) standard on the third grade reading and mathematics tests, and obtained the 
highest mean scale scores on the 2015 STAAR reading and mathematics tests. 


Should you have any questions or require any further information, please contact Carla Stevens 
in the Department of Research and Accountability, at 713-556-6700. 
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Head Start Program Longitudinal Effects Study: Third Grade Follow-Up, 
2014-2015 Third Grade STAAR Reading and Mathematics Performance 


By Lai Kwan Pei, Ph.D. 


The goal of Head Start is to develop the cognitive and social-emotional skills of children from low-income families 
to prepare them to succeed in kindergarten and beyond. Presently, Houston Independent School District (HISD) 
collaborates with four federally-funded Head Start agencies: AVANCE, Gulf Coast Community Services Association 
(GCCSA), Harris County Department of Education (HCDE), and Neighborhood Centers, Inc. (NCI). Each Head 
Start agency provides, not only high-quality educational programs to 3- or 4-year-old low income children, but also 
offers access to health, dental, and other support services in order to meet families’ needs. The purpose of this report 
was to explore the lasting impact of Head Start on students’ performance on the third grade STAAR reading and 
mathematics tests. The findings suggested that economically-disadvantaged students who attended Head Start 
programs obtained higher mean scale scores, and had higher percentages of students who met the 2015 STAAR Level 
II: Satisfactory (Phase-In 1) standard on both third grade reading and mathematics tests than their peers in the 
district. Among the four Head Start agencies, NCI had the highest percentage of students who met the 2015 STAAR 
Level II: Satisfactory (Phase-In 1) standard on the third grade reading and mathematics tests, and obtained the 


highest mean scale scores on the 2015 STAAR reading and mathematics tests. 


Background 


Head Start programs are publicly-funded and managed 
at the local level but must adhere to federal quality 
guidelines. These guidelines suggest that Head Start 
agencies provide a learning environment that promotes 
cognitive and social-emotional development to enhance 
school-readiness of low-income students. In addition, 
Head Start agencies are expected to provide a wide array 
of social services to assist families. The Head Start 
programs are expected to: 1) allow students to 
experience a more integrated school day with in-depth 
study of pre-k curriculum, 2) promote school readiness, 
and 3) contribute to the narrowing of achievement gaps 
related to school readiness at the start of kindergarten 
and subsequent grade levels (Gormley, Gayer, & 
Phillips, 2005). 


Presently, the Houston Independent School District 
(HISD) collaborates with four federally-funded Head 
Start agencies: AVANCE, Gulf Coast Community 
Services Association (GCCSA), Harris County 


Department of Education (HCDE), and Neighborhood 
Centers, Inc. (NCI). 


Review of the Literature 


Past evaluations of Head Start programs suggest that a 
Head Start intervention can have both short-term and 
long-term benefits for children. For example, short-term 
benefits include improvements in cognitive and 
achievement outcomes (Shager et al., 2013). Longer- 
term benefits of Head Start include a reduction in the 
likelihood of special education placement and a 
reduction in the incidence of early grade retention. In 
addition, some studies have found that a quality Head 
Start intervention increases the likelihood of high school 
graduation and achievement test score gains (Currie, 
2001; Currie & Neidell, 2007). 


The short-term benefits of four Head Start agencies 
(AVANCE, GCCSA, HCDE, and NCI) have been 
evaluated in other reports that found that Head Start 
programs had positive impact on economically- 
disadvantaged students. The goal of this study was to 


measure the lasting effect of the four Head Start 
programs on students’ 2014-2015 STAAR reading and 
mathematics performance at third grade. Specifically, 
this study compared the third grade STAAR reading and 
mathematics performance of a cohort of students who 
attended one of four Head Start programs in 2010-2011. 


Variations in findings regarding the benefits of Head 
Start sometimes have to do with methodological 
differences and the selection of comparison groups 
(Zhai, Brooks-Gunn, & Waldfogel, 2011; Shager et al., 
2013). Previous studies have compared students who 
received a formal preschool education to all other 
students who did not receive a formal preschool 
education without controlling for demographic 
characteristics, such as economic status, that influence 
student performance (Gormley et al., 2005). Given the 
negative effects of low socio-economic status on 
academic outcomes (e.g., Aikens & Barbarin, 2008; 
Brooks-Gunn, 2003, 2005; Chatterji, 2006), this report 
has taken into consideration students’ demographic 
characteristics when comparing Head Start students’ 
performance on the 2014-2015 STAAR reading and 
mathematics tests. 


Measures 


Student performance data were collected through the 
STAAR (regular English version) reading and 
mathematics tests. STAAR is the state of Texas 
criterion-referenced assessment, and it replaced the 
Texas Assessment of Knowledge and Skills (TAKS) 
program in spring 2012. The Texas Education Agency 
(TEA), in collaboration with the Texas Higher Education 
Coordinating Board (THECB) and Texas educators, 
developed this new assessment system in response to 
requirements set forth by the 80th, 81st and 83rd Texas 
legislatures. This new system focuses on increasing 
postsecondary readiness of graduating high school 
students, and helps to ensure that Texas students are 
competitive with other students both nationally and 
internationally. 


The key outcome measures for this report were the 
2014-2015 STAAR reading and mathematics scale 
scores of third grade students. The 2014-2015 STAAR 
Level II: Satisfactory (Phase-in I) performance standard 
was also used to measure the proportion of students who 
met the standard in reading and mathematics. 


Methods 


In order to examine the effects of the four Head Start 
agencies on student’s third grade performance, a cohort 
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of students who attended one of the four Head Start 
programs in 2010-2011 was tracked up to the end of 
third grade. The descriptive statistics (mean scale scores 
and percentages of students who met STAAR Level II: 
Satisfactory (Phase-in I) standard on the 2014—2015 
STAAR reading and mathematics tests were used to 
describe the lasting impact of four Head Start agencies 
on students enrolled in the programs in 2010-2011 and 
student subgroups (ethnicity, gender, economically- 
disadvantaged, special education placement, limited 
English proficiency (LEP), and at-risk status). 


Aikens and Barbarin (2008) suggested that economic 
status has a strong effect on students’ performance, and 
other factors, such as LEP and at-risk status are also 
associated with student performance. Thus, student 
groups were disaggregated by ethnicity, gender, 
economically-disadvantaged, special education 
placement, LEP, and at-risk status to control for the 
effect of student demographic characteristics on the 
students’ academic performance on third grade STAAR. 


Sample 


The study sample was students who attended one of the 
four Head Start programs in 2010-2011, and took the 
third grade STAAR test in 2014-2015. Only students 
who had 2015 STAAR reading and mathematics scores 
were included in this study. Consequently, the sample 
size of this study was 1,429. The demographic 
characteristics of students are shown in Appendix - 
Table 1. For AVANCE, 153 students were identified as 
2014-2015 HISD third graders; for GCCSA, 583 
students; for HCDE, 82 students; and for NCI, 611 
students. Notably, in the four Head Start agencies, the 
majority of students (over 90%) were economically- 
disadvantaged, and 75.9% of the students were Hispanic 
based on student information at the time that the student 
enrolled in HISD third grade in 2014—2015. 


How did Head Start students enrolled in the four 


Head Start agencies in 2010-2011 perform on the 
2014-2015 third grade STAAR reading test? 


The impact of the Head Start program on HISD third 
grade students’ performance was measured using the 
STAAR reading and mathematics results. The 2015 third 
grade STAAR mean scale scores in reading for students 
who attended one of the four Head Start agencies in 
2010-2011 are displayed in Figure 1. Appendix -Table 
2 presents the number of students who took the third 
grade STAAR reading test in 2014~—2015, and the means 
and standard deviations of the scale scores by the four 
Head Start agencies and by other student groups 


(ethnicity, gender, economically-disadvantaged, special 
education placement, LEP, and at-risk status). 

Figure 1 shows that in the overall sample, Head Start 
students who were enrolled in one of the four Head Start 
programs in 2010-2011 (M = 1398.5) scored lower than 
the district mean scale score (M = 1412.0) on the 2014— 
2015 STAAR reading test by 13.5 points. 


Among the four Head Start agencies, students from NCI 
(M = 1408.1) obtained the highest mean scale score. The 
mean scale score of HCDE (M = 1363.1) was lower than 
students from the other three Head Start agencies on the 
2014-2015 STAAR reading test (Figure 1). 
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Figure 1. Mean scale scores on the 2014-2015 third grade STAAR 
reading test for Head Start students who were enrolled in 2010-2011. 


The percentages of Head Start students who were 
enrolled in 2010-2011 Head Start programs and met the 
2015 STAAR Level II: Satisfactory (Phase-In 1) reading 
standard are displayed in Figure 2. Appendix -Table 3 
presents the number of students who took the third grade 
STAAR reading test in 2014—2015, and the percentage 
of Head Start students who met the STAAR Level II: 
Satisfactory (Phase-In 1) reading standard by the four 
Head Start agencies and by other student groups 
(ethnicity, gender, economically-disadvantaged, special 
education placement, LEP, and at-risk status). 


Figure 2 shows that in the overall sample, 70.2% of Head 
Start students met the 2015 STAAR Level II: 
Satisfactory (Phase-In 1) reading standard compared to 
69.0% for the district. 


Among the four Head Start agencies, NCI (73.7%) had 
the highest percentage of students who met the 2015 
STAAR Level II: Satisfactory (Phase-In 1) standard on 
the reading test, which was higher than the district 
percentage (Figure2). 


GCCSA (69.8%) had comparable percentages of 
students who met the 2015 STAAR Level II: Satisfactory 
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(Phase-In 1) standard on the reading test as the district 
percentage (Figure 2). 
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Figure 2. Percentage of Head Start students who were enrolled in 
2010-2011 Head Start programs and met the 2015 third grade STAAR 
Level II: Satisfactory (Phase-In 1) reading standard. 


How did Head Start students enrolled in the four 


Head Start agencies in 2010-2011 perform on the 


2014-2015 third grade STAAR mathematics test? 


The 2014-2015 third grade STAAR mean scale scores 
in mathematics for students who attended one of the four 
Head Start agencies’ programs in 2010-2011 are 
displayed in Figure 3. Appendix -Table 4 presents the 
number of students who took the third grade STAAR 
mathematics test in 2014-2015, and the means and 
standard deviations of the scale scores by the four Head 
Start agencies and by other student groups (ethnicity, 
gender, economically-disadvantaged, special education 
placement, LEP, and at-risk status). 


Figure 3 shows that in the overall sample, Head Start 
students who were enrolled in one of the four Head Start 
programs in 2010-2011 (M = 1429.3) scored lower than 
the district mean scale score (M = 1438.0) on the 2014— 
2015 STAAR mathematics test by 8.7 points. 


Among the four Head Start agencies, students from NCI 
(M = 1443.8) obtained the highest mean scale score, 
which was higher than the district mean scale score. 
Students from AVANCE (M = 1425.9) and GCCSA (M 
= 1420.2) obtained comparable mean scale scores, which 
were lower than the district mean scale score. The mean 
scale score of HCDE (M = 1394.7) was lower than 
students from the other three Head Start agencies on the 
2014-2015 STAAR mathematics test (Figure 3). 
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Figure 3. Mean scale scores on the 2014—2015 third grade STAAR 
mathematics test for Head Start students who were enrolled in 2010— 
2011. 


The percentages of Head Start students who were 
enrolled in 2010-2011 Head Start programs and met the 
2015 STAAR Level II: Satisfactory (Phase-In 1) 
mathematics standard are displayed in Figure 4. 
Appendix -Table 5 presents the number of students who 
took the third grade STAAR mathematics test in 2014— 
2015, and the percentage of Head Start students who met 
the 2015 STAAR Level II: Satisfactory (Phase-In 1) 
mathematics standard by the four Head Start agencies 
and by other student groups (ethnicity, gender, 
economically-disadvantaged, special education 
placement, LEP, and at-risk status). 


Figure 4 shows that in the overall sample, 72.1% of Head 
Start students met the 2015 STAAR Level II: 
Satisfactory (Phase-In 1) mathematics standard 
compared to 71.0% for the district. 


Among the four Head Start agencies, NCI (77.5%) had 
the highest percentage of students who met the 2015 
STAAR Level II: Satisfactory (Phase-In 1) standard on 
the mathematics test, which was higher than the district 
percentage (Figure2). 


GCCSA (68.4%) and AVANCE (67.9%) had fairly 
comparable percentages of students who met the 2015 
STAAR Level II: Satisfactory (Phase-In 1) standard on 
the mathematics test (Figure 4). 
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Figure 4. Percentage of Head Start students who were enrolled in 
2010-2011 Head Start programs and met the 2015 third grade STAAR 
Level II: Satisfactory (Phase-In 1) mathematics standard. 


How did economically-disadvantaged Head Start 


students enrolled in the four Head Start agencies in 
2010-2011 perform on the 2014-2015 third grade 
STAAR reading and mathematics tests? 


Figure 5 shows the economically-disadvantaged Head 
Start students (M = 1396.0) had a higher mean scale 
score than the district (M = 1383.0) on the 2014-2015 
STAAR reading test. 


Economically-disadvantaged Head Start students (M = 
1430.3) had a higher mean scale score than the district 
(M = 1410.0) on the 2014-2015 STAAR mathematics 
test (Figure 5). 
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Figure 5. Mean scale scores on the 2014-2015 STAAR reading and 
mathematics tests for economically-disadvantaged Head Start students 
who were enrolled in 2010-2011. 


Figure 6 shows that on the 2014-2015 STAAR reading 
test, the percentage of economically-disadvantaged 
Head Start students (69.9%) who met the STAAR Level 
II: Satisfactory (Phase-In 1) reading standard was higher 
than the district percentage (64.0%). 


On the 2014-2015 STAAR mathematics test, the 
percentage of economically-disadvantaged Head Start 
students (71.6%) who met the STAAR Level II: 
Satisfactory (Phase-In 1) mathematics standard was 
higher than the district percentage (66.0%) (Figure 6). 


mHead Start (Economically-disadvantaged students) 


= District Average (Economically-disadvantaged students) 


60 


— 
oI 
mo} 
S 
is] 
yn 
pe 
@ 
= 
s 


STAAR Reading STAAR Mathematics 
Figure 6. Mean scale scores on the 2014-2015 STAAR reading and 
mathematics tests for economically-disadvantaged Head Start students 
who were enrolled in 2010-2011. 


Discussion 


The goal of Head Start programs is to provide learning 
opportunities to economically-disadvantaged students to 
help them develop and maintain foundational skills 
necessary to be successful in school. This report 
explored the lasting impact of Head Start programs on 
HISD third graders by analyzing student academic 
performance on the 2014-2015 STAAR reading and 
mathematics tests. 


The findings suggested that the impact of Head Start 
programs on student third grade academic performance 
was evident for both STAAR reading and mathematics 
scores, especially for the economically-disadvantaged 
students. When student performance was compared by 
Head Start agency, the findings revealed that students 
who attended NCI obtained higher mean scale scores 
than the other agencies on both STAAR reading and 
mathematics tests. NCI also had the higher STAAR 
passing rates than the district in both reading and 
mathematics tests before accounting for economically 
disadvantaged status. 
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Economically-disadvantaged students who attended 
Head Start programs outperformed their economically- 
disadvantaged peers in the district. Consequently, the 
economically-disadvantaged students may have 
acquired significant academic benefits from the Head 
Start program and maintained that academic edge to third 
grade. This finding was contrary to many previous 
studies (Barnett & Hustedt, 2005). The literature 
suggested that the effect of preschool usually diminishes 
over time. However, the findings in this report show that 
the beneficial effect of Head Start persists at least into 
third grade for economically-disadvantaged students. 


In this report, the third grade STAAR test scores were 
used to evaluate the impact of Head Start on students’ 
academic performance because the end of third grade has 
been found to detect an effect in cumulative measures of 
school success. Future studies on the lasting impact of 
Head Start programs should examine other aspects of 
school success, such as special education placement, and 
disciplinary incidents. 
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Department of Research and Accountability at 713- 


556-6700 or e-mail Research@Houstonisd.org. 
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Appendix 


Table 1. Demographic Characteristics of HISD 3 Grade 


Students Who were 2010-2011 Head Start Cohort 


Student Group Percent 


=} 


Agency 


GCCSA 583 40.8 


NCI 611 42.8 

Male 753 52.7 

African- 321 22.5 

White 9 0.6 
Economically No 133 9.3 
Special No 1337 93.6 
Limited English No 583 40.8 
Proficient (LEP) 


At-Risk No 223 15.6 


Note. The demographic information used in this table was based on student information at 
the time that the student enrolled in HISD 3" grade in 2014-2015. 
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Table 2. Mean Scale Scores on the 2014-2015 STAAR Reading Test 


Student Group Mean SD n 
Agency 
GCCSA 1395.7 143.2 526 
NCI 1408.1 142.6 547 
Gender 
1386.6 138.6 658 
Ethnicity 
African- 1383.6 135.9 274 
White 1322.3 78.5 8 
Economically No 1422.8 138.6 121 
Special No 1402.0 140.1 1,229 
Limited No 1398.7 135.4 494 
English 
profcent 
At-Risk No 1495.6 111.9 206 


Note. 1.* denotes fewer than 5 students, and were not reported. 2. The demographic information 
used in this table was based on student information at the time that the student enrolled in HISD 
3 grade in 2014-2015. 
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Table 3. Percentage of Students Who Met the 2015 STAAR Level II: 


Satisfactory (Phase-In 1) Reading Standard 


Student Group % n 
Overall Sample 70.2% 897 
Agency 
GCCSA 69.8% 367 
NCI 73.7% 403 
Male 66.9% 440 
African- 63.5% 174 
American 


+ 
+ 


White 


Economically No 73.6% 89 
Special No 71.4% 878 
Limited English No 68.6% 339 
Proficient (LEP) 


At-Risk No 94.2% 


a= 
oO 
KK 


Note. 1.* denotes fewer than 5 students, and were not reported. 2. The demographic information 
used in this table was based on student information at the time that the student enrolled in HISD 
3 grade in 2014-2015. 
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Table 4. Mean Scale Scores on the 2014-2015 STAAR Mathematics Test 


Student Group Mean SD n 
Overall Sample 1429.3 138.1 1,267 
Agency 
GCCSA 1420.2 136.9 525 
Cn eae 
NCI 1443.8 135.9 538 
Gender 


1429.6 140.5 651 


Ethnicity 
African- 1378.7 137.1 274 
White 1416.3 66.5 7 
Economically No 1420.6 138.7 120 
Special No 1432.6 137.9 1,221 
Limited No 1406.1 141.3 495 
English 
Proficient 
(LEP) 
At-Risk No 1493.3 131.9 206 


Note. 1.* denotes fewer than 5 students, and were not reported. 2. The demographic information 
used in this table was based on student information at the time that the student enrolled in HISD 
3" grade in 2014-2015. 
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Table 5. Percentage of Students Who Met the 2015 STAAR Level II: 


Satisfactory (Phase-In 1) Mathematics Standard 


Student Group % n 
Overall Sample 72.1% 913 
Agency 
GCCSA 68.4% 359 
NCI 77.5% 417 
Male 72.2% 470 
African- 59.1% 162 
White 100.0% 7 
Economically No 76.7% 92 
Special No 73.1% 893 
Limited English No 66.3% 328 
Proficient (LEP) 


At-Risk No 86.9% 179 


Note. 1.* denotes fewer than 5 students, and were not reported. 2. The demographic information 
used in this table was based on student information at the time that the student enrolled in HISD 
3" grade in 2014—2015. 
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